Big Data Analytics Overview

Principles

Definition

Big data analytics is a methodological approach and research field dedicated to examining extremely large, complex, and diverse datasets to uncover patterns, correlations, hidden insights, and other valuable information. This process involves employing advanced analytical techniques and computational tools to address challenges associated with data volume, velocity, variety, and veracity, thereby enabling data-driven decision-making and discovery across various domains.

Ontological type

Core Methods

Applications

Computing Architectures

Key developments

Key Figures

Scalable Analytics Consolidation era

Jeffrey Dean and Sanjay Ghemawat anchored scalable analytics with MapReduce and Bigtable, enabling large-scale data processing that underpinned analytics pipelines. Doug Cutting and the Hadoop ecosystem extended access to large-scale processing, while Hive and Pig translated SQL-like queries into MapReduce workflows, broadening adoption. Matei Zaharia and the AMPLab community advanced cloud-aware scheduling with Mesos, enabling multiple analytics engines to share clusters and paving the way for iterative analytics. Avinash Lakshman and Prashant Malik highlighted the NoSQL wave with Dynamo-inspired distributed stores, and Dan Abadi's C-Store/H-Store work demonstrated columnar layouts that optimize distributed scans and compression for analytics.

Deep Learning Integration era

Geoffrey Hinton, Yoshua Bengio, and Yann LeCun defined the era by strengthening deep learning foundations and enabling scalable neural representations across diverse data. Andrew Ng bridged research and industry, championing end-to-end pipelines and deployment of real-time analytics on large, heterogeneous data. Jeff Dean and the Google Brain team advanced distributed training, TensorFlow, and data-centric infrastructures that make big data analytics feasible at scale. Ian Goodfellow and Fei-Fei Li contributed to model-centric value through generative modeling and large-scale labeled datasets, promoting reproducibility and cross-domain benchmarking.

Sustainability-Driven Analytics era

In Sustainability-Driven Analytics (2021-2024), representative authors include Thomas H. Davenport, Michael E. Porter, and Cathy O'Neil, whose work intersects big data analytics, sustainability strategy, and governance. Davenport has emphasized integrating enterprise-scale analytics into operational decisions to improve resource efficiency and to generate transparent sustainability metrics across green supply chains. Porter, drawing on the shared value framework, has shaped how analytics platforms quantify environmental and social outcomes alongside economic performance to guide corporate sustainability strategy. O'Neil's focus on accountability and ethics of algorithms provides governance criteria for green AI and data-sharing infrastructures that underlie circular-economy metrics.